Method and Apparatus for Monitoring Medication Adherence

ABSTRACT

A system and method for positioning a pill to be ingested by a user in a field of view of an image acquisition camera. The method includes the steps of determining a desired location of the mouth of a user in a field of view of an image acquisition camera, determining a current position of a pill to be ingested by the user in the field of view of the image acquisition apparatus, and indicating on a display a movement to be made by the user to move the pill towards mouth of the user.

FIELD OF THE INVENTION

This invention relates generally to a method and apparatus for assistingmonitoring of, and for improving medication adherence, and moreparticularly to an apparatus and method for providing feedback to a userof a gesture recognition system for monitoring of medication adherencerelated to positioning the user, a medication, or other object within animage acquisition area, and to provide positive feedback related to theposition of the user, the medication and the like in order to encourageand improve adherence to medication administration protocols and toshape patient behavior in a positive manner. The invention may also actto insure and aid in the ability for proper acquisition of variousgesture sequences and images when applied to an image acquisition systempotentially having a narrow field of view, or a fixed direction camera.Failure on any of these fronts may result in additional instruction orencouragement from the system, in either an automated or personalmanner, and may be provided as input data to an overall populationmanagement system.

BACKGROUND OF THE INVENTION

Gesture recognition systems typically include high sensitivity camerasincluding a wide field of view, multiple cameras, tracking systems, orany combination of the above. Such systems typically require customsystem hardware systems to allow for proper image acquisition. Whenbeing employed to assist in monitoring medication adherence, it may notbe possible or convenient to provide such a specialized device to eachuser.

Employing gesture recognition as well as object recognition and trackingto monitor medication adherence may result in the monitoring ofindividuals and medication. These individuals and the medication theyare holding or administering may be improperly positioned within a fieldof view of the camera. While the higher quality gesture recognitionsystems noted above may be able to capture necessary images in spite ofsuch variability, standard camera systems that may be provided in astandard computer or laptop configuration may not be sufficient forperforming gesture recognition. Furthermore, mobile devices, typicallyincluding an even lower resolution camera and lower computing power mayhave more difficulty in implementing such a gesture recognition system.

Therefore, it would be desirable to provide an apparatus and system thatallow for proper gesture recognition in such a context, even employinglower quality hardware components and processing power, and thatovercomes other drawbacks of the prior art.

SUMMARY OF THE INVENTION

In accordance with various embodiments of the invention a gesturerecognition assistance method and apparatus for assisting in monitoringmedication adherence may be provided. Such an assistance method mayinclude a directional system for insuring that a user is properlypositioned within an image acquisition space. The directional system maypreferably include a color and/or shape based indication system todirect a user to a location in an image acquisition space to insureproper image acquisition. Alternative embodiments of the invention mayfurther include sound to aid in conveying information to a user. Thedirectional system may also be employed to determine and instructchanges in location of a pill or other medication. In accordance with apreferred embodiment of the system, when implemented on a mobile deviceor other reduced processing power device, a user may attempt to havetheir image acquired by the device including a fixed, relatively lowresolution camera, a user may be shown an image or other representationof themselves and including various indications of incorrectpositioning. The user may further be shown various arrows indicating adirection in which to move. Further, color may be employed to give afurther indication to the user of proper location.

Therefore, by providing such guidance to the user, not only is the userguided through a sequence of steps to aid in proper medicationadministration, by effectively reducing the variability in userbehavior, the processing required to accurately determine whether theuser is properly administering such medication is also reduced.

Furthermore, the present invention may also be applicable to additionalforms of medication administration, such as injectables, inhalers,topical applications, ocular applications and the like, as well asvarious other medical maintenance procedures, such as user of cardiacmonitors, or other monitoring devices or the like. Indeed, the inventionmay be applicable to any medical procedure in which reducing variabilityof patient behavior to allow for remote monitoring employing a gesturerecognition system may be employed.

Still other objects and advantages of the invention will in part beobvious and will in part be apparent from the specification anddrawings.

The invention accordingly comprises the several steps and the relationof one or more of such steps with respect to each of the others, and theapparatus embodying features of construction, combinations of elementsand arrangement of parts that are adapted to affect such steps, all asexemplified in the following detailed disclosure, and the scope of theinvention will be indicated in the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the invention, reference is made tothe following description and accompanying drawings, in which:

FIG. 1 depicts a user being directed to move closer to a display andimage capture apparatus in accordance with an embodiment of theinvention;

FIG. 2 depicts a user indicated as situated as being a proper distancefrom the display and image capture apparatus in accordance with anembodiment of the invention;

FIG. 3 depicts a user being directed to move laterally in relation to adisplay and image capture apparatus in accordance with an embodiment ofthe invention;

FIG. 4 depicts a position target in accordance with an embodiment of theinvention;

FIG. 5 depicts a position indicator in accordance with an embodiment ofthe invention;

FIG. 6 depicts a position indicator indicating that a desired object hasbeen identified in accordance with an embodiment of the invention;

FIG. 7 depicts a position indicator and target indicating that theposition indicator has reached the desired target to provide positivereinforcement to a user in accordance with an embodiment of theinvention;

FIG. 8 depicts various positioning locations of a position indicator inrelation to the display and image acquisition apparatus to provide anindication of position relative to the camera and mouth of the user inaccordance with an embodiment of the invention;

FIG. 9 depicts a position indicator indicating that a desired object hasbeen identified in relation to a mouth position indicator target inaccordance with an embodiment of the invention;

FIG. 10 depicts the position indicator of FIG. 9 in a different positionrelative to the mouth position indicator target of FIG. 9 in accordancewith an embodiment of the invention;

FIG. 11 depicts the position indicator of FIG. 9 and the mouth positionindicator of FIG. 9 indicating that the position indicator has reachedthe desired mouth position indicator target position in accordance withan embodiment of the invention; and

FIG. 12 depicts a display and image capture apparatus that may beemployed in accordance with one or more embodiments of the presentinvention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The invention will now be described making reference to the followingdrawings in which like reference numbers denote like structure or steps.Referring first to FIG. 12, a display and image capture apparatus 1200that may be employed in accordance with one or more embodiments of thepresent invention is shown. Apparatus 1200 preferably includes a housing1210, an image capture camera 1220 and a display 1230. Image capturecamera 1220 may comprise a single camera element, a stereo cameraelement or other appropriate image capture device. Other elements, asknown to one of ordinary skill in the art, including ports, powerattachments, processors for processing data, and the like may also beprovided in accordance with apparatus 1200. It is further contemplatedin accordance with the various embodiments of the invention thatapparatus may include one or more self-check mechanisms, includingmechanisms for determining proper ambient light, direction andbackground of the camera and the background imaged by the camera, orother environmental issues that may be improved to further aid in theimaging of any images by apparatus 1200. Additionally, if at any time itis determined that lighting conditions are too difficult for imagingapparatus 1200, it may be possible to provide a light burst from astrobe or display to aid in illuminating the images to be captured. Thefollowing descriptions of the various embodiments of the invention willassume that such a display and image capture apparatus 1200 is employed.The invention, however, shall not be so limited and may be employed onany structure of image capture camera and display, whether contained ina single or multiple apparatuses.

Referring next to FIG. 1, a face of a user 110 is shown on display 1230of display and image capture apparatus 1200. As can be seen in FIG. 1,this face is shown as being small, an indication of being greater than adesired distance from an imaging apparatus. This situation of being toofar away may be further accentuated by a small head tracing 120 beingapproximately the size of the user's head image. It has been determinedby the inventors of the present invention that proper positioning of theface of a user improves the ability to perform various gesturerecognition procedures, and in particular aids in reducing variabilityin action that must be processed by image capture apparatus 1200.Therefore, by reducing variability in position, and in turn thereforevariability in action, gesture recognition for determining medicationadherence may be performed in an easier manner. Furthermore, proper facepositioning may be even more important when one considersreduced-resolution cameras. Thus, if the user is positioned too far,perhaps not enough pixels may be imaged to properly perform such gesturerecognition, while being positioned too close may result in imagecapture device not being able to image the entire face of the user.Further, if lighting conditions are poor, the user may be asked to movecloser to aid in image capture. Thus, proper positioning may also beprovided as a method for addressing variations in camera resolution,processing power, lighting or other environmental conditions, and thelike.

Also provided may be an intermediate sized head tracing 130 showing adirection in which the user is to move, and a full sized head tracing140. Once the user's head is sized coincident with the full sized headtracing, it may be considered that the user is at a correct distancefrom the image capture camera and display, as is shown in FIG. 2. It isfurther contemplated that if the user is positioned closer than desiredto image capture camera 1220, the user image may be shown on display1230 as larger than the full sized head tracing 140. In either case, asis shown in FIG. 1, the desired sized head tracing 140 may be shown in adifferent line format than the other head tracings, in this exemplarycase, a dotted line for the full sized head tracing 140, solid lines forthe others. Of course any desired line formats may be used. Further, inaddition to providing such head tracings, an avatar or other human orother indicator indicating a direction in which the user is to move maybe provided on display 1230. Further, the incorrect head tracings may beprovided in a first color, such as red, while the correct head tracingmay be provided in a second color, such as green in order to furtherprovide user feedback and an easy to recognize indication of correct orincorrect action. Any desirable combination of color and line format maybe employed.

Referring next to FIG. 3, once a user is positioned a proper distancefrom display and image capture apparatus 1200 (or, alternatively, whilethe user is still not positioned a proper distance from the display andimage capture apparatus 1200, the present embodiment being described forclarity), he or she may be positioned incorrectly in a lateral directionto a desired position. Thus as is shown, display 1230 may show an imageof user 110 and may include a full sized head tracing 220 indicating acurrent position of the user (of course, the smaller and larger headtracings of FIG. 1 may be provided here if the user is also an incorrectdistance from display and image capture apparatus 1200), and a desiredposition full head tracing 240. As shown, the head tracing and thedesired head tracing are preferably drawn employing different lineformats to indicate the desired position. An arrow 230 may also beprovided indicating a direction in which the user is to move. As withFIG. 1, an avatar or other human or other indicator may also be providedon display 1230, indicating a direction in which the user is to move.Further, the incorrect head tracings may be provided in a first color,such as red, while the correct head tracing may be provided in a secondcolor, such as green. In addition, the same red or other color may beused to illuminate a side of the image if the user moves off the screenin a particular direction, the side of movement being illuminated ondisplay 1230.

Next, FIGS. 4-7 depict an embodiment of the invention for locating asingle or group of objects, and in accordance with this embodiment ofthe invention such an object may comprise a pill or other medication tobe ingested by a user. Thus, as is shown in FIG. 4, a circular indicator410 may be provided on display 1230 indicative of a first position thatthe user is to place the object or in this situation, a pill. Of course,for this circular indicator and any other indicators described inaccordance with the invention, any desired indicators may be employed.The invention may be similarly applicable to other medicationadministration procedures, employing for example, injectables, inhalers,topical applications, ocular applications and the like, as well asvarious other medical maintenance procedures, such as user of cardiacmonitors, or other monitoring devices or the like, or any other medicaldevice that may benefit from remote, automated monitoring. As is thenshown in FIG. 5, a pill 505 is shown adjacent indicator 410, as is shownby the outer ring proximity indicator 510. Once the pill is positionedconcentric with the indicator 410 and the proximity indicator, the pillis shown in a target bulls eye 610, including an outer ring and a centerindicator. To properly position pill 505, target bulls eye 610 as theuser moves pill 505, target bulls eye moves in display 1230 in a similarmanner. Thus, movement in both the lateral direction and in thedirection towards and away from image capture apparatus 1200 may bemonitored, and feedback provided to the user. For example, as the usermoves pill 505 closer to display 1230, the pill and indicator 410 maygrow larger, while movement away from the display may cause theindicator to shrink. Thus, in addition to assisting the user in placingthe pill properly laterally, the indicator system may also aid inpositioning the pill a proper distance from the image capture apparatus1200. Concentric larger rings 710 (as shown in FIG. 7) may be positionedon display 1230 to indicate a desired position to which the user is tomove pill 505. When the target bulls eye is positions concentric tolarger rings 710, and is indicated as being a correct size to fittherein, thus providing a concentric set of circles on display 1230, itcan be determined by the user that pill 505 has been properly positionedin the various desired directions. As with the earlier embodiments,arrows or other directional indicators may be provided, and color may beused to aid in directing the user. Thus, a red color may be used toindicate a side of an image to which the pill may have been moved offthe screen.

Referring next to FIG. 8, if a user is positioning a pill, or is toposition a pill on the display, indicator 410 may be shown on display1230 in various locations vertically in order to account for relativevertical positioning of the image acquisition camera relative to theuser. Thus, if the camera is positioned above the user, the user will beencouraged to hold the pill at a higher location, thus allowing for theimaging of the pill and user at the same time. Similarly, if the camerais positioned lower than the user, the user will be encouraged to holdthe pill at a lower location, thus being positioned approximatelybetween the camera and the user's face, as shown by the lower indicator410 in FIG. 8. Thus, indicator 410 may be positioned in any locationalong the arrow shown in FIG. 8 to ensure proper pill location to aid inproper imaging of the user and the pill. Such positioning further aidsin reducing the possibility of occlusion of the pill or other medicationby the hand of the user, or other possible occlusion, and additionallyaids in keeping the pill or other medication within the field of view ofimage capture camera 1220.

Referring next to FIG. 9, a method and system for aiding in confirmingpill ingestion by a user in accordance with an embodiment of theinvention is shown. Once again, a pill 505 is included under a targetbulls eye 610 as originally portrayed in FIG. 6. Also shown in alocation target 910, similar to the concentric large rings 710 of FIG.7. Location target 910, however, is preferably provided to determine andindicate a location of the user's mouth on display 1230. Therefore, inaccordance with the invention, the user's mouth location is determinedin accordance with the image acquisition camera, and thereafter,location target 910 may be provided on display 1230 coincidenttherewith. As is further shown in FIG. 10, as the user moves pill 505,target bulls eye 610 follows along, indicating a relative position ascompared with location target 910 on display 1230. Various techniquesnoted above, including arrows, color and the like may be employed inaiding the user in positioning pill 505 concentric with location target910. Of course, as the position of location target 910 is preferablycoextensive with the user's mouth, movement of the pill to the user'smouth may result in the desired interaction between pill 505 andlocation target 910, thus encouraging the desired action. It is throughthis continual monitoring that the user is further encouraged to want toperform actions correctly. As if being watched, the automated systemprovided in accordance with various embodiments of the present inventionprovides positive reinforcement to a user, and also may provided theadded incentive of consistent and continuous monitoring. Rather thanrequiring an actual person to continuously review the actions of theuser, automatic monitoring and feedback may provide a similar userresponse without the need for additional personnel. Furthermore, such anautomated system may provide even greater consistency in monitoring,thus further encouraging desired behavior and further reducingvariability in user action.

Once properly positioned, as is shown in FIG. 11, properly positionedlocator 615 is shown in a differently displayed location target 915,thus indicating successful positioning. As noted above, color change,change in line texture or the like may be used to indicate that the userhas properly positioned the pill relative to their mouth, thus providingpositive feedback to the user upon proper action having been taken.

Therefore, in accordance with the invention, a set of one or moreindicators may be provided to a user to aid in proper following of apredetermined script to aid in the proper acquisition of the variousmovements of the user, so that the system is able to determine whetherthe user is properly positioned in front of an image acquisition camera,and whether the user has properly positioned a pill for ingestion.Various positive reinforcement mechanisms may be employed in accordancewith the monitoring system of the various embodiments of the invention,thus encouraging the user to properly perform various actions andreducing the variability in such actions, therefore easing the burden onthe system to properly determine proper performance of such actions bythe user. The user may also perceive the system as one that is easy touse, thus even further encouraging its use.

It will thus be seen that the objects set forth above, among those madeapparent from the preceding description, are efficiently attained and,because certain changes may be made in carrying out the above method andin the construction(s) set forth without departing from the spirit andscope of the invention, it is intended that all matter contained in theabove description and shown in the accompanying drawings shall beinterpreted as illustrative and not in a limiting sense.

It is also to be understood that this description is intended to coverall of the generic and specific features of the invention hereindescribed and all statements of the scope of the invention which, as amatter of language, might be said to fall there between.

1-20. (canceled)
 21. A method for positioning a medication relative to auser in a field of view of an image acquisition camera by providingautomatic guidance to the user on how to position the medication, themethod comprising the steps of: displaying on a display an image of auser within a field of view of an image acquisition camera; presenting alocation target on the display at a desired location of a portion of abody of the user indicative of a desired position relative to a portionof a body of the user in the field of view of the image acquisitioncamera; acquiring an image of a medication within the field of view ofthe image acquisition camera by the image acquisition camera; presentingthe medication image on the display indicative of a current position ofthe medication relative to the position of the user; indicating on thedisplay a movement to be made by the user to move the current positionof the medication relative to the image acquisition camera so as to movethe medication image on the display towards the location target on thedisplay; following movement of the medication in the field of view ofthe image acquisition camera by relative movement of the medicationimage on the display; and confirming interaction between the medicationand the desired location of the portion of a body of the user when thelocation target and medication image are determined to be coextensive onthe display.
 22. The method of claim 21, wherein if the current positionof the user is too far from the image acquisition camera, an image ofthe user is displayed on the display in a small size.
 23. The method ofclaim 22, further comprising the step of providing a head tracingindicating a proper size that the image of the user should be.
 24. Themethod of claim 23, wherein an indicator is provided when the user isproperly positioned, and the image of the user is coincident with thehead tracing.
 25. The method of claim 24, wherein the step of indicatingcomprises a change in color.
 26. The method of claim 21, wherein thedesired location of the user is adjusted in accordance with one or moreenvironmental factors.
 27. The method of claim 21, wherein if thecurrent position of the medication relative to the desired location ofthe user is laterally incorrect, the user is shown on the display anindication of desired movement of the medication image towards thelocation target, indicative of a direction in which to move themedication within the field of view of the image acquisition camera. 28.The method of claim 27, further comprising the step of providing a headtracing indicating a desired proper location of the image of the user.29. The method of claim 28, wherein the indicator directs the user tomove in the direction of the head tracing.
 30. A method for positioninga pill to be ingested by a user in a field of view of an imageacquisition camera, comprising the steps of: displaying on a display animage of a user within a field of view of an image acquisition camera;presenting a location target on the display indicative of a desiredposition of the mouth of the user in the field of view of the imageacquisition camera; presenting a pill image on the display indicative ofa current position of a pill to be ingested relative to the position ofthe mouth of the user; indicating a movement to be made by the user tomove the pill relative to the image acquisition camera so as to move thepill image on the display towards the location target on the display;following movement of the pill in the field of view of the imageacquisition camera by relative movement of the pill indicator on thedisplay; and confirming interaction between the pill and mouth of theuser when the location target and pill image are determined to becoextensive on the display.
 31. The method of claim 30, wherein if thecurrent position of the pill relative to the desired location of themouth of the user is laterally incorrect, the user is shown adirectional indicator on the display indicative of a direction in whichto move the pill within the field of view of the image acquisitioncamera.
 32. The method of claim 31, wherein the directional indicatordisplayed on the display directs the user to move the pill in thedirection of the target so that the pill image moves in the direction ofthe location target on the display.
 33. The method of claim 32, furthercomprising the step of indicating if the pill leaves the field of viewof the image acquisition camera.
 34. A system for tracking positioningof a medication to be administered, comprising: an image acquisitionapparatus for acquiring an image of a patient within a field of view ofthe image acquisition apparatus, and for acquiring an image of amedication to be administered within the field of view of the imageacquisition apparatus; a processor for determining a position of adesired location of a portion of a body part of the patient to which themedication is to be administered in the field of view of the imageacquisition apparatus, and determining a current position of themedication to be administered to the patient in the field of view of theimage acquisition apparatus; and a display for displaying a locationtarget at a desired location of the portion of the body part of thepatient in the acquired image of the patient to receive the medication,a medication image indicative of a current position of the medication inthe acquired image of the medication and relative to the locationtarget, and for indicating a desired movement of the medication to movethe medication image towards the location target on the display so thatthe medication is moved towards the portion of the body part of thepatient to which the medication is to be administered; wherein theprocessor tracks movement of the medication, and the display displaysmovement of the medication indicator in accordance with the trackedmovement of the medication; and wherein the processor confirmsinteraction between the medication and the portion of the body part ofthe patient to which the medication is to be administered when thelocation target and the medication image are determined to becoextensive on the display.
 35. The system of claim 34, whereinprovision of the indicated movement improves the ability of a user toproperly administer the medication.
 36. The system of claim 34, whereinthe display further displays an indication if the medication to beadministered moves out of the field of view of the image acquisitionapparatus and is therefore no longer displayed on the display.
 37. Thesystem of claim 36, wherein the indication includes the use of a colordisplayed on the display.
 38. The system of claim 34, wherein the imageacquisition apparatus acquires a plurality of images of the patient andof the medication to track movement thereof.
 39. The system of claim 38,wherein interaction between the pill and mouth of the user is confirmedwhen the location target and pill image are determined to be coextensivein one or more of the plurality of images.